This invention relates to compression-coding of image signals and decoding of the compression-coded signals, and more particularly relates to compression-coding of a high definition television (HDTV) signal and decoding of the HDTV signal for conversion into a standard resolution television signal.
It is known to compression-code image signals and then to record the compression-coded signals on a recording medium such as an optical disk, a magneto-optical disk or a magnetic tape and then to reproduce, decompress and display the recorded signal.
FIG. 21 is a block diagram of a previously proposed apparatus in which compression-coded image signals are recorded, and the recorded signals are reproduced and decompressed. In the system shown in FIG. 21, an input image signal such as, for example, a high definition television signal, originating from a signal source (not shown) such as a television camera, a video tape recorder, a video disk player, etc., is supplied to a preprocessing circuit 1. The preprocessing circuit 1 separates the input image signal into a brightness signal component (Y) and a color difference signal component (C). An analog-to-digital (A/D) converter 2 receives the brightness signal component Y and converts it into a digital signal which is stored in a frame memory 4. At the same time, an A/D converter 3 receives the color difference signal C and converts it into a digital signal which is stored in a frame memory 5.
A reformatting circuit 6 converts the frames of digital brightness and color difference signals stored in the frame memories 4 and 5 into data blocks. The resulting data blocks are provided to an encoder circuit 7 which encodes the blocks of data and outputs a bit stream for recording on a recording medium 8, which may be, for example, an optical disk, a magneto-optical disk or a magnetic tape.
A bit stream of data reproduced from the recording medium 8 is provided to a decoder circuit 9, which decodes the bit stream and outputs blocks of decoded data to a format conversion circuit 10. The format conversion circuit 10 converts the blocks of data into frames of brightness and color difference signals which are respectively stored in a brightness signal frame memory 11 and a color difference signal frame memory 12. The brightness data stored in the frame memory 11 is provided to a digital-to-analog (D/A) converter 13 for conversion into an analog brightness signal, while the color difference data stored in the frame memory 12 are provided to a D/A converter 14 for conversion into an analog color difference signal. The resulting analog brightness and color difference signals are composed by a post-processing circuit 15 to form an output image signal which may be, for example, displayed on a monitor (not shown), rerecorded, transmitted, etc.
The encoding performed by encoder circuit 7 involves compressing the blocks of image data so that the number of bits required for recording the same is reduced. According to conventional compression-coding methods, line correlation or inter-frame correlation within the image signal are used to accomplish compression. When line correlation is used, discrete cosine transform (DCT) processing, or the like, is performed in order to compress the data. Moreover, inter-frame correlation can be used to provide compression by coding of difference signals. For example, referring to FIG. 22, three succeeding frames PC1, PC2 and PC3 are shown, the same having been generated at respective times t1, t2 and t3. A picture PC12 can be generated by calculating the difference between the pictures PC1 and PC2, and the picture PC23 can be generated by calculating the difference between the pictures PC2 and PC3. Only a relatively small amount of information is required to represent the difference pictures PC12 and PC23, so that coding of signals representing the difference pictures rather than the pictures PC2 and PC3 permits significant data compression. However, it is not possible to reconstitute the entire picture just from the difference pictures. Therefore, it has been proposed to perform compression coding with difference signals by forming each frame of the image signal as one of three types of pictures: an I-picture, a P-picture and a B-picture.
To provide a particular example, and referring to FIG. 23(A), seventeen frames F1 to F17 of an image signal are processed together as a group. The first frame F1 is coded as an I-picture; the second frame F2 is processed as a B-picture; and the third frame F3 is processed as a P-picture. Thereafter, the fourth through seventeenth frames are alternately processed as a B-picture or a P-picture. For the frame which is processed as an I-picture, all of the image data representing the frame are coded for transmission, recording or the like (hereinafter, "transmit" and "transmission" should be understood to include, for example, recording of a signal on a recording medium and reproduction of the recorded signal, as well as transmission through a transmission channel such as a cable, a satellite broadcast system, or a conventional over-the-air broadcast system). For each frame that is processed as a P-picture, the data to be transmitted is the difference between the data for the present frame and the data for the preceding I-picture or P-picture frame, as is illustrated in FIG. 23(A). The data transmitted with respect to each B-picture is calculated as the difference formed by subtracting the image data for the current B-picture frame from an average value formed from the preceding and following frames, as illustrated in FIG. 23(B).
A process of compression coding using inter-frame correlation is schematically illustrated in FIG. 24.
As to the first frame F1, all of the image data is coded for transmission, without reference to other frames, resulting in so-called intra-picture coding.
As to the next frame F2, which is processed as a B-picture, there are four alternative ways in which coding may be performed. The first is simply intra-picture coding as for an I-picture so that a signal indicated as SP1 is transmitted. The second way in which the B-picture frame may be processed is on the basis of the difference between that frame and the following frame F3, resulting in production of a transmission signal SP2. This is referred to as backward predictive coding. The third way in which a B-picture may be processed is based on the difference between the frame F2 and the preceding frame. This produces the transmission signal SP3 and is known as forward predictive coding. Finally, the fourth way in which the B-picture frame may be coded is on the basis of the difference from an average value calculated based on the preceding and following frames. The resulting transmission signal is indicated as SP4 and the method is referred to as bi-directional predictive coding. Whichever one of the four methods results in the minimum amount of transmission data is adopted, and the resulting data is transmitted as transmission data frame F2X. If a processing method other than intra-picture coding is used, then at least one of a motion vector x1 representing motion between the current frame and the previous frame, and a motion vector x2 representing motion between the current frame the next frame, are generated and transmitted with the difference data. In the case of forward prediction coding, the motion vector x1 is transmitted; in the case of backward predictive coding, the motion vector x2 is transmitted; and both are transmitted if bi-directional predictive coding is used.
Regarding the frame F3, which is processed as a P-picture, either intra-picture coding, or forward predictive coding is used, depending on which of those methods results in the smaller amount of data to be transmitted.
Referring to FIG. 25, it will be explained how a frame made up of V lines in a vertical direction and H dots in a horizontal direction is divided into macro blocks. The frame is divided into N slices in the vertical direction and each slice is divided into M macro blocks in the horizontal direction, each macro block consisting of a 16.times.16 array of picture elements. For each macro block there are formed four 8.times.8 blocks Y 1! to Y 4! of brightness data, which together represent all of the 16.times.16 picture elements in the macro block. At the same time, two 8.times.8 data blocks Cb 5! and Cr 6! representing color difference signals are included in each macro block. With data reduction and time base multiplexing, only an 8.times.8 block of data is used to represent each type of color difference signal for the 16.times.16 picture elements of the macro block. Within each slice, the image data is arranged macro block by macro block, and within each macro block, the image data is arranged in the 8.times.8 blocks in raster scanning order. The order in which the blocks Y 1! to Y 4!, Cb 5! and Cr 6! are transmitted is indicated by the numbers that are part of the respective symbols.
High resolution image data, such as data representing an HDTV signal, can be thinned out by a factor of one half in each of the vertical and horizontal directions in order to obtain lower resolution image data. If the aspect ratio of the resulting image is changed from 16:9 to 4:3, then the image can be displayed using a conventional NTSC system television receiver.
FIG. 26 illustrates a decoder that may be used for converting high resolution image data into image data having one-quarter of the resolution (i.e., with the resolution reduced by one-half in each of the horizontal and vertical directions). Such a decoder may be used as the decoder 9 shown in FIG. 21. For the purposes of the decoder shown in FIG. 26, it is assumed that the image data provided thereto resulted from compression-coding by means of DCT processing.
Referring to FIG. 26, the input image signal is in the form of a stream of coefficient data elements which were obtained by performing DCT processing upon 8.times.8 blocks of picture element data. The input image data stream is provided to the sampling circuit 21, which forms it into 8.times.8 blocks of coefficient data as shown in FIG. 27. In FIG. 27, the data elements toward the bottom of the array correspond to higher frequency components of the picture in the vertical direction and the data elements toward the right hand side of the array correspond to higher frequency components of the picture in the horizontal direction.
Referring again to FIG. 26, the 8.times.8 blocks of coefficient data elements are provided to a sampling circuit 22, which forms from those blocks 4.times.4 blocks of coefficient data elements as shown in FIG. 28. The resulting 4.times.4 blocks are simply the upper left hand quadrant of the 8.times.8 block shown in FIG. 27.
The resulting 4.times.4 block of coefficient data elements is provided to an inverse discrete cosine transform (IDCT) circuit 23, which performs an inverse discrete cosine transform upon the 4.times.4 block, thereby providing picture element data in which the resolution has been reduced by one-half in both the vertical and horizontal directions. In an apparatus as just described, when lower resolution image data is to be obtained from high resolution image data, only the coefficient data elements corresponding to low frequency components are used. As a result, the interlace structure represented by high frequency components is lost so that the low resolution picture produced by the conventional apparatus fails to present motion smoothly. Further, when predictive coding is used in a conventional system as described above in which DCT processing is applied on the encoder side to 8.times.8 blocks of picture elements and on the decoder side 4.times.4 blocks of picture elements are provided from the decoded data in order to provide a lower resolution picture, the predictive picture produced on the decoder side does not entirely coincide with a predictive picture produced on the encoder side for the purposes of forming difference signals. As a result, when the predictive picture produced on the decoder side is used for decoding difference signals, mismatch errors accumulate, resulting in loss of interlace structure and deterioration in quality of the reproduced low resolution picture.